Papers with Transformer-based method
Neural Document Segmentation Using Weighted Sliding Windows with Transformer Encoders (2025.coling-industry)
Copied to clipboard
| Challenge: | Using overlapping text sequences and position-aware weighting, we achieve up to a 10% increase in segmentation F1 score compared to existing methods. |
| Approach: | They propose a Transformer-based method for document segmentation that utilizes overlapping text sequences with a unique position-aware weighting mechanism to enhance segmentation accuracy. |
| Outcome: | The proposed method achieves up to 10% increase in segmentation F1 score compared to existing methods and improves quality of generated responses by 5% while achieving four times greater efficiency. |
Transformer-based Lexically Constrained Headline Generation (2021.emnlp-main)
Copied to clipboard
Kosuke Yamada, Yuta Hitomi, Hideaki Tamori, Ryohei Sasano, Naoaki Okazaki, Kentaro Inui, Koichi Takeda
| Challenge: | Existing automatic headline generation methods cannot include a given phrase in the generated headline. |
| Approach: | They propose a Transformer-based method that guarantees to include a given phrase in a generated headline. |
| Outcome: | The proposed method achieves ROUGE scores comparable to previous methods with Japanese news corpus. |
CoAD: Automatic Diagnosis through Symptom and Disease Collaborative Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Automated diagnosis (AD) is a critical application of AI in healthcare . despite its simplicity and superior performance, a decline in disease diagnosis accuracy is observed . |
| Approach: | They propose a new collaborative disease and symptom generation framework to improve automatic diagnosis. |
| Outcome: | The Transformer-based method achieves an average 2.3% improvement over previous state-of-the-art methods . it can be used to query patients about their symptoms and health concerns . |
Does Structure Matter? Encoding Documents for Machine Reading Comprehension (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing Transformer-based models for machine reading comprehension treat documents as flat sequences. |
| Approach: | They propose a Transformer-based method that reads a document as tree slices and jointly trains and consults the modules at inference time. |
| Outcome: | The proposed method outperforms several baseline approaches on two datasets from varied domains. |
Recurrent Neural Networks with Mixed Hierarchical Structures and EM Algorithm for Natural Language Processing (2022.lrec-1)
Copied to clipboard
| Challenge: | A variety of hierarchical RNN models have been proposed to incorporate hierarchically-based hierarchic information in modeling languages in the literature. |
| Approach: | They propose a latent indicator layer approach to identify and learn hierarchical information and develop an EM algorithm to handle the latent indicators layer in training. |
| Outcome: | The proposed approach outperforms other RNN-based models in document classification tasks. |